1. Field of the Invention
Embodiments of the present invention relate to processing and reporting statistical data.
2. Background Art
Statistical data becomes more useful when it is sorted or grouped before it is reported. Some statistical data may be sorted based on levels of data granularity. For instance, data may be sorted according to broad categories. The data within each broad category may then be sorted according to narrower categories. This sorting may proceed to even more narrow or specific categories or groupings. At some point, any further granularity may not be useful. Such sorting may take place after aggregating statistics from one or many locations.
Once data has been aggregated and sorted, it is often reported to a computer or a human observer for analysis or evaluation. Such reporting may reflect a grouping of data records performed by a computer. This earlier grouping may have helped to limit the size of the data. The grouping may also have compiled any data that would not be very interesting to a user nor would have contributed to his or her analysis. For example, consider a program that provides information regarding business activity, or sales, for a company. Business activity in smaller towns may not result in a significant level of revenue compared to larger cities. When there are many cities to be observed, these data points compete for the attention of an observer. The observer may prefer that the revenues of these smaller towns be added up and placed in a general category representing the total revenue from smaller towns in the state. That is, information is provided only for towns above a certain size, with the remaining towns being grouped into a single record. The data records for the smaller towns would be compiled into a single data record representative of the compiled data. Compilation could then be performed as necessary for broader categories such as states or provinces within a country. The compilation of data may be particularly useful when handling a large amount of data or a large amount of streaming data.
Handling large amounts of streaming data can consume a lot of computer resources. Many present systems for grouping and reporting streaming data use a memory-map based approach. That is, all data is loaded into memory before compiling data records and reporting the data. Such an approach consumes a significant amount of resources and presents scaling difficulties.